Papers with autoregressive generation tasks
Mitigating Exposure Bias in Grammatical Error Correction with Data Augmentation and Reweighting (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to grammatical error correction (GEC) use sequence-to-sequence models, but there is an exposure bias problem. |
| Approach: | They propose a data manipulation approach to overcome the exposure bias problem in seq2seq GEC . they propose augmentation methods to mimic decoder input and reweighting methods to automatically balance the importance of each kind of augmented samples. |
| Outcome: | The proposed method improves on benchmark GEC datasets. |
XLM-D: Decorate Cross-lingual Pre-training Model as Non-Autoregressive Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-training language models have been successful in natural language understanding and autoregressive generation tasks, but non-autoregressive models have not been sufficiently successful. |
| Approach: | They propose a pre-trained masked language model (MLM) and a non-autoregressive generation model with a lightweight decorator. |
| Outcome: | The proposed model outperforms the previous mask-predict model on translation datasets by 19.9x. |